Contrastive learning (CL), which can extract the information shared between different contrastive views, has become a popular paradigm for vision representation learning. Inspired by the success in computer vision, recent work introduces CL into graph modeling, dubbed as graph contrastive learning (GCL). However, generating contrastive views in graphs is more challenging than that in images, since we have little prior knowledge on how to significantly augment a graph without changing its labels. We argue that typical data augmentation techniques (e.g., edge dropping) in GCL cannot generate diverse enough contrastive views to filter out noises. Moreover, previous GCL methods employ two view encoders with exactly the same neural architecture and tied parameters, which further harms the diversity of augmented views. To address this limitation, we propose a novel paradigm named model augmented GCL (MA-GCL), which will focus on manipulating the architectures of view encoders instead of perturbing graph inputs. Specifically, we present three easy-to-implement model augmentation tricks for GCL, namely asymmetric, random and shuffling, which can respectively help alleviate high- frequency noises, enrich training instances and bring safer augmentations. All three tricks are compatible with typical data augmentations. Experimental results show that MA-GCL can achieve state-of-the-art performance on node classification benchmarks by applying the three tricks on a simple base model. Extensive studies also validate our motivation and the effectiveness of each trick. (Code, data and appendix are available at https://github.com/GXM1141/MA-GCL. )

Estimating the structure of directed acyclic graphs (DAGs) of features (variables) plays a vital role in revealing the latent data generation process and providing causal insights in various applications. Although there have been many studies on structure learning with various types of data, the structure learning on the dynamic graph has not been explored yet, and thus we study the learning problem of node feature generation mechanism on such ubiquitous dynamic graph data. In a dynamic graph, we propose to simultaneously estimate contemporaneous relationships and time-lagged interaction relationships between the node features. These two kinds of relationships form a DAG, which could effectively characterize the feature generation process in a concise way. To learn such a DAG, we cast the learning problem as a continuous score-based optimization problem, which consists of a differentiable score function to measure the validity of the learned DAGs and a smooth acyclicity constraint to ensure the acyclicity of the learned DAGs. These two components are translated into an unconstraint augmented Lagrangian objective which could be minimized by mature continuous optimization techniques. The resulting algorithm, named GraphNOTEARS, outperforms baselines on simulated data across a wide range of settings that may encounter in real-world applications. We also apply the proposed approach on two dynamic graphs constructed from the real-world Yelp dataset, demonstrating our method could learn the connections between node features, which conforms with the domain knowledge.

In the field of antibody engineering, an essential task is to design a novel antibody whose paratopes bind to a specific antigen with correct epitopes. Understanding antibody structure and its paratope can facilitate a mechanistic understanding of its function. Therefore, antibody structure prediction from its sequence alone has always been a highly valuable problem for de novo antibody design. AlphaFold2, a breakthrough in the field of structural biology, provides a solution to predict protein structure based on protein sequences and computationally expensive coevolutionary multiple sequence alignments (MSAs). However, the computational efficiency and undesirable prediction accuracy of antibodies, especially on the complementarity-determining regions (CDRs) of antibodies limit their applications in the industrially high-throughput drug design. To learn an informative representation of antibodies, we employed a deep antibody language model (ALM) on curated sequences from the observed antibody space database via a transformer model. We also developed a novel model named xTrimoABFold to predict antibody structure from antibody sequence based on the pretrained ALM as well as efficient evoformers and structural modules. The model was trained end-to-end on the antibody structures in PDB by minimizing the ensemble loss of domain-specific focal loss on CDR and the frame-aligned point loss. xTrimoABFold outperforms AlphaFold2 and other protein language model based SOTAs, e.g., OmegaFold, HelixFold-Single, and IgFold with a large significant margin (30+\% improvement on RMSD) while performing 151 times faster than AlphaFold2. To the best of our knowledge, xTrimoABFold achieved state-of-the-art antibody structure prediction. Its improvement in both accuracy and efficiency makes it a valuable tool for de novo antibody design and could make further improvements in immuno-theory.

大多数图形神经网络（GNN）通过学习输入图和标签之间的相关性来预测看不见的图的标签。但是，通过对具有严重偏见的训练图进行图形分类调查，我们发现GNN始终倾向于探索伪造的相关性以做出决定，即使因果关系始终存在。这意味着在此类偏见的数据集中接受培训的现有GNN将遭受概括能力差。通过在因果观点中分析此问题，我们发现从偏见图中解开和去偏置因果和偏见的潜在变量对于偏见至关重要。在此鼓舞下，我们提出了一个普遍的分解GNN框架，分别学习因果子结构和偏见子结构。特别是，我们设计了一个参数化的边蒙版生成器，以将输入图明确分为因果和偏置子图。然后，分别由因果/偏见感知损失函数监督的两个GNN模块进行培训，以编码因果关系和偏置子图表中的相应表示。通过分离的表示，我们合成了反事实无偏的训练样本，以进一步脱离因果变量和偏见变量。此外，为了更好地基于严重的偏见问题，我们构建了三个新的图形数据集，这些数据集具有可控的偏置度，并且更容易可视化和解释。实验结果很好地表明，我们的方法比现有基线实现了优越的概括性能。此外，由于学习的边缘面膜，该拟议的模型具有吸引人的解释性和可转让性。代码和数据可在以下网址获得：https：//github.com/googlebaba/disc。

图形神经网络（GNN）已被广泛用于建模图形结构化数据，这是由于其在广泛的实用应用中令人印象深刻的性能。最近，GNNS的知识蒸馏（KD）在图形模型压缩和知识转移方面取得了显着进步。但是，大多数现有的KD方法都需要大量的真实数据，这些数据在实践中不容易获得，并且可能排除其在教师模型对稀有或难以获取数据集培训的情况下的适用性。为了解决这个问题，我们提出了第一个用于图形结构化数据（DFAD-GNN）的无数据对抗知识蒸馏的端到端框架。具体而言，我们的DFAD-GNN采用生成性对抗网络，主要由三个组成部分组成：预先训练的教师模型和学生模型被视为两个歧视者，并利用生成器来衍生训练图来从教师模型进入学生模型。在各种基准模型和六个代表性数据集上进行的广泛实验表明，我们的DFAD-GNN在图形分类任务中显着超过了最新的无数据基线。

建议图表神经网络（GNNS）在不考虑训练和测试图之间的不可知分布的情况下，诱导GNN的泛化能力退化在分布外（OOD）设置。这种退化的根本原因是大多数GNN是基于I.I.D假设开发的。在这种设置中，GNN倾向于利用在培训中存在的微妙统计相关性用于预测，即使它是杂散的相关性。然而，这种杂散的相关性可能在测试环境中改变，导致GNN的失败。因此，消除了杂散相关的影响对于稳定的GNN来说是至关重要的。为此，我们提出了一个普遍的因果代表框架，称为稳定凝球。主要思想是首先从图数据中提取高级表示，并诉诸因因果推理的显着能力，以帮助模型摆脱虚假相关性。特别是，我们利用图形池化层以提取基于子图的表示作为高级表示。此外，我们提出了一种因果变量区别，以纠正偏置训练分布。因此，GNN将更多地集中在稳定的相关性上。对合成和现实世界ood图数据集的广泛实验良好地验证了所提出的框架的有效性，灵活性和可解释性。

尽管图形神经网络（GNNS）已经取得了显着的准确性，但结果是否值得信赖仍未开发。以前的研究表明，许多现代神经网络对预测过度充满信心，然而，令人惊讶的是，我们发现GNN主要呈相反方向，即，GNN是不受自信的。因此，非常需要GNN的置信度校准。在本文中，我们通过设计拓扑知识的后HOC校准函数提出了一种新型值得信赖的GNN模型。具体而言，我们首先验证图形中的置信度分布具有同眼性的财产，而且这一发现激发了我们设计校准GNN模型（CAGCN）以学习校准功能。 CAGCN能够从GNN的Logits对每个节点的校准置信度获得独特的变换，同时，这种变换能够在类之间保留课程之间的顺序，满足精度保留的属性。此外，我们将校准GNN应用于自培训框架，表明可以通过校准的置信度获得更可靠的伪标签，并进一步提高性能。广泛的实验证明了我们所提出的模型在校准和准确性方面的有效性。

Graph neural network, as a powerful graph representation technique based on deep learning, has shown superior performance and attracted considerable research interest. However, it has not been fully considered in graph neural network for heterogeneous graph which contains different types of nodes and links. The heterogeneity and rich semantic information bring great challenges for designing a graph neural network for heterogeneous graph. Recently, one of the most exciting advancements in deep learning is the attention mechanism, whose great potential has been well demonstrated in various areas. In this paper, we first propose a novel heterogeneous graph neural network based on the hierarchical attention, including node-level and semantic-level attentions. Specifically, the node-level attention aims to learn the importance between a node and its metapath based neighbors, while the semantic-level attention is able to learn the importance of different meta-paths. With the learned importance from both node-level and semantic-level attention, the importance of node and meta-path can be fully considered. Then the proposed model can generate node embedding by aggregating features from meta-path based neighbors in a hierarchical manner. Extensive experimental results on three real-world heterogeneous graphs not only show the superior performance of our proposed model over the state-of-the-arts, but also demonstrate its potentially good interpretability for graph analysis.

The development of social media user stance detection and bot detection methods rely heavily on large-scale and high-quality benchmarks. However, in addition to low annotation quality, existing benchmarks generally have incomplete user relationships, suppressing graph-based account detection research. To address these issues, we propose a Multi-Relational Graph-Based Twitter Account Detection Benchmark (MGTAB), the first standardized graph-based benchmark for account detection. To our knowledge, MGTAB was built based on the largest original data in the field, with over 1.55 million users and 130 million tweets. MGTAB contains 10,199 expert-annotated users and 7 types of relationships, ensuring high-quality annotation and diversified relations. In MGTAB, we extracted the 20 user property features with the greatest information gain and user tweet features as the user features. In addition, we performed a thorough evaluation of MGTAB and other public datasets. Our experiments found that graph-based approaches are generally more effective than feature-based approaches and perform better when introducing multiple relations. By analyzing experiment results, we identify effective approaches for account detection and provide potential future research directions in this field. Our benchmark and standardized evaluation procedures are freely available at: https://github.com/GraphDetec/MGTAB.

Image Virtual try-on aims at replacing the cloth on a personal image with a garment image (in-shop clothes), which has attracted increasing attention from the multimedia and computer vision communities. Prior methods successfully preserve the character of clothing images, however, occlusion remains a pernicious effect for realistic virtual try-on. In this work, we first present a comprehensive analysis of the occlusions and categorize them into two aspects: i) Inherent-Occlusion: the ghost of the former cloth still exists in the try-on image; ii) Acquired-Occlusion: the target cloth warps to the unreasonable body part. Based on the in-depth analysis, we find that the occlusions can be simulated by a novel semantically-guided mixup module, which can generate semantic-specific occluded images that work together with the try-on images to facilitate training a de-occlusion try-on (DOC-VTON) framework. Specifically, DOC-VTON first conducts a sharpened semantic parsing on the try-on person. Aided by semantics guidance and pose prior, various complexities of texture are selectively blending with human parts in a copy-and-paste manner. Then, the Generative Module (GM) is utilized to take charge of synthesizing the final try-on image and learning to de-occlusion jointly. In comparison to the state-of-the-art methods, DOC-VTON achieves better perceptual quality by reducing occlusion effects.